Style Based Search – Using RNN for searching Web Design Gallery

Current search engines are largely text based, and the text vocabulary that they use is a poor match for the style based concerns. Search engines return pages whose content relates semantically to the keyword query; they cannot be used for searches along stylistic attributes such as “pages with funny backgrounds” or “pages that look professional” etc. We propose a system1 that will allow users to query on stylistic attributes. The search query could be either language-keyword based or it could be based on examples i.e. a user can select a design and ask the system to show other examples in the corpus that are similar (or dissimilar) to the designs in the query. For example, a user can select a a web page, and query the system for other pages with similar (or dissimilar) layout and design. In this paper we describe how a Recursive Neural Network (RNN) can be used to build such systems. Given a set of training examples consisting of web pages and the corresponding style label distributions, we learn the parameters of the system via a back-propagation neural network which is applied recursively to each node in the binary tree representation of the web page. Once these parameters are learned, the system can predict the style distribution for any given web page. Example-based querying can be implemented on top of this representation. A comparative study shows that RNN works better than other ML techniques in context of this problem. Author